Efficiency in Games with Markovian Private Information

We study repeated Bayesian games with communication and observable actions in which the players’ privately-known payoffs evolve according to an irreducible Markov chain whose transitions are independent across players. Our main result implies that, generically, any Pareto-efficient payoff vector above a stationary minmax value can be approximated arbitrarily closely in a perfect Bayesian equilibrium as the discount factor goes to one. As an intermediate step we construct an approximately efficient dynamic mechanism for long finite horizons without assuming transferable utility.